53  Validity and Internal Validity

54 Validity

54.1 Introduction

‘Validity’ in research refers to the extent to which a method, technique, or test measures what it claims to measure.

There are five different types of validity:

Validity is a benchmark that determines the credibility and reliability of our outcomes. We want to conduct valid research that contributes to the body of knowledge in our field by providing results that are both trustworthy and applicable in the ‘real world’.

By thinking carefully about validity, we ensure that the conclusions drawn from our analysis are truly reflective of the phenomena being investigated, and not the result of extraneous variables or flaws in our methodology.

55 Internal Validity

55.1 Understanding Internal Validity

‘Internal validity’ refers to the extent to which the results of a study can be attributed to the variables it’s intended to test, rather than external influences or confounding variables. It’s particularly important in experimental research, where we’re trying to explore cause-and-effect relationships.

High internal validity ensures that the changes we observe in the dependent variable (DV) are due to the manipulation of the independent variable (IV), and not other factors. This is vital for drawing accurate and reliable conclusions from the study.

High internal validity gives us confidence that the observed effects are genuinely caused by the variables under investigation.

55.2 Key Characteristics of Internal Validity

The key characteristics of internal validity include:

  • Control of confounding variables: we need to control and manage variables that could potentially influence the outcome of the study, ensuring that any observed effects can be attributed to the independent variable.

  • Proper randomisation: we use random assignment in experimental groups to minimise the effects of participant characteristics that could otherwise affect the results.

  • Consistent procedures: we prioritise uniformity in the application of procedures, treatments, and measurements throughout the study to ensure that variations in the dependent variable are due to the independent variable.

  • Temporal precedence: we establish a clear chronological order in which the cause (independent variable) precedes the effect (dependent variable), reinforcing the causal relationship.

  • Appropriate operationalisation of variables: we clearly define and measure our variables in a way that accurately captures the theoretical constructs they represent.

55.3 Threats to Internal Validity

Unfortunately, lots of things can threaten the internal validity of a study, and potentially compromise the accuracy of our conclusions.

These include:

  • History: external events occurring during the course of the study can affect the outcomes, independent of the experimental manipulations.

  • Maturation: changes within our participants can occur over time, such as growing older or gaining more experience, which in turn can influence the study results.

  • Testing: the effects of taking a test more than once, where the first test influences the results of subsequent tests, rather than the experimental manipulation.

  • Instrumentation: changes in measurement tools or procedures over time might affect the consistency of the data collected.

  • Statistical regression: the tendency for extreme scores to move towards the mean upon retesting, which can be mistaken for an effect of the experimental manipulation.

  • Selection bias: we need to be aware of differences in the type of participants selected for different groups in a study, which can influence the results independently of the experimental treatment.

  • Experimental mortality (attrition): loss of participants from the study over time, which can lead to unequal groups and biased results if the attrition is not random.

  • Selection-maturation interaction: occurs when changes within the participants interact with the selection, such as when one group matures at a different rate compared to another due to initial group differences.

55.4 Strategies to Enhance Internal Validity

To enhance internal validity, we can use several strategies to mitigate potential threats.

One strategy is the use of control groups, which provides a baseline by allowing comparisons between subjects who received the treatment and those who did not.

  • This approach helps in isolating the effects of the treatment from other variables. Random assignment of participants to control and experimental groups is another critical strategy, as it helps ensure that the groups are equivalent at the start of the experiment, reducing the risk of selection bias.

Also, we can implement blinding or masking techniques where participants, and sometimes the experimenters, are unaware of who received the treatment, minimising biases from expectations or the placebo effect.

  • Blind designs can be single-blind, where either the participant or the experimenter is unaware of the treatment allocation, or double-blind, where both are unaware.

Another approach is to conduct pilot studies or preliminary testing. These help in identifying and rectifying potential problems in study design and implementation before conducting the full-scale study.

  • Pilot studies can reveal issues with the experimental setubp, measurement tools, or participant understanding that could impact internal validity.

Longitudinal designs, where the same participants are observed over an extended period, can be used to control for maturation effects. They allow us to observe changes over time and distinguish them from the effects of the experimental manipulation.

Repeated measures, where the same tests or observations are made on all participants under all conditions, can also be beneficial. This approach controls for individual differences among participants and reduces variability that can obscure the treatment effects.

Statistical control methods, such as covariance analysis, can adjust for the effects of confounding variables, helping to isolate the impact of the independent variable.

We can also enhance internal validity by carefully defining and operationalising our variables, ensuring that they accurately reflect the constructs under study.

Lastly (but importantly), transparency in reporting and a thorough explanation of methodologies used in research can aid in enhancing internal validity.

  • By clearly describing the procedures, the rationale behind choices made in the study design, and the methods used to control or account for potential threats to validity, we create opportunities for a better assessment of our study’s validity. It also facilitates replication by other researchers.